Accuracy in pharmacoeconomic literature review: lessons learned from the Navajo Code Talkers.

The Navajo Code Talkers, who faithfully transmitted messages in an indecipherable code to support U.S. military efforts from 1942 to 1945, are widely credited with the achievement of numerous victories in the Pacific theater during World War II (WWII), among them the capture of Iwo Jima by the Marine Corps in 1945.1-3 Originally a highly classified military secret, the Code Talkers’ work was completely unknown to the American public—and even to the Code Talkers’ own families—until its declassification in 1968.4 Official recognition of the Code Talkers’ accomplishments did not come until 1992, a half-century after their work began.2 The hallmark of the Code Talkers’ work was a rapid but exacting translation process. Receiving messages in “a string of seemingly unrelated Navajo words,” each Code Talker first translated the words to English, then used the first letter of each English word to spell out an English message.2 In addition, both for security purposes and to speed translation, the Code Talkers memorized a dictionary of an astonishing 450 Navajo words representing commonly used military terms; for example, the word “besh-lo” (iron fish) referred to a submarine, “dah-he-tih-hi” (hummingbird) was a fighter plane, and “debeh-li-zine” (black street) indicated a squad.2,3 Because of the innate complexity of the Navajo language, the use of multiple Navajo words to represent a single English letter, and the detailed nature of the translation process, the code was both impossible for anyone else to decipher and painstakingly difficult for the Code Talkers themselves. “When we were in the [training] classroom we were drilled and drilled,” one Code Talker recalled in a 2005 interview. “No writing it down. It was all memorized. ... At the end of the class you had to hand in every pencil and piece of paper.”1 Yet, despite the difficulty of the work and the pressures of performing it during combat, the Code Talkers consistently turned in error-free performances. During the first 2 days of the battle of Iwo Jima, 6 Code Talkers worked around the clock to transmit 800 messages, all with 100% accuracy.2 With some dismay, JMCP editors have recently noted that an increasing number of literature reviews, both in published work and in manuscripts submitted to us, bear little or no resemblance to the careful translations that characterized the heroic cryptographers of WWII. It is common for us to read in a submitted manuscript a statement that “in Disease A, Drug X is widely accepted as more efficacious than Drug Y,” only to find in our research that the source cited for the statement investigated a different disease state, studied only a handful of people, produced a finding that Drug X was not superior to Drug Y, or otherwise did not support the alleged claim. A case in point is found in examination of the literature published in the years following Soumerai et al.’s 1991 often-cited study of the effect of a medication coverage limit on a sample of chronically ill elderly Medicaid enrollees in New Hampshire.5 A comparison of the original study report with descriptions of the study that were provided in later manuscripts provides a revealing and at times disturbing look at the uses—and abuses—of medical literature applied in the service of making a point. This editorial reviews a sample of those descriptions and suggests a direction for the future.